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ABSTRACT 


An experimental investigation of the Deformable Mirror 
Device (DMD) developed by Texas Instruments at Dallas for 
use in object identification was completed. The DMD was 
tested as a joint correlator. The DMD was used as a spatial 
light modulator on which the squared modulus of the Fourier 
transform of test object pairs was written. The squared 
modulus was phase encoded on the DMD after it had been 
thresholded and rewritten as a binary phase function. The 
thresholding was found to produce a sharp peak in the 
autocorrelation when the test objects were matched and no 
significant peak in the case of distinct objects. It was 
concluded that the use of the DMD as a joint correlator 
looks promising and further studies should be carried out. 
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INTRODUCTION 


* 


The location and recognition of objects belonging to 
small well defined sets ( e.g., handtools drifting in space) 
or objects belonging to large less well defined classes 
(e.g., boulders on the surface of Mars.) are typical parts 
of mission requirements on several NASA programs including 
the planned unmanned expedition to Mars, the EVA (Extra 
Vehicular Activity) retriever program and autonomous 
rendezvous and docking. In most cases the speed of the 
recognition process must rival that of the human eye-brain. 
This is difficult competition. Even the fastest available 
digital computers would not be able to process data fast 
enough to keep pace with human vision. An interesting 
alternative to digital computing is optical computing. 
Optical computing takes advantage of massive parallel 
processing. Within the last 10 years, optical processing 
techniques have matured to the point that they are worthy of 
consideration in tasks where speed is a premium. Although it 
lacks the flexibility of digital processing at the present, 
it can perform certain fixed tasks, such as Fourier 
transforms in two dimensions, almost instantaneously (i.e., 
essentially in the length of time it takes the light to pass 
through the optical processing system.) At present, the 
Tracking and Communications group at NASA is studying the 
use of a hybrid system that exploits the best of both 
digital and optical processing for artificial vision 
problems. The key element in this approach is the Deformable 
Mirror Device (DMD) , a new type of integrated optics device 
under development by Texas Instruments (TI) in Dallas. 

The DMD consists of a 1/4" square array containing 128 
x 128 mirror elements. Each mirror element consists of a 
set of four individual cantilevered mirrors arranged as in a 
2X2 matrix (roughly in the form of a cloverleaf.) Each of 
the mirror elements can be deflected as a group using the 
electrostatic forces determined by capacitatively stored 
charge between a mirror and the array back plane. The charge 
stored, and thus the deflection, is determined by on-chip 
electronics and addressing. The DMD is. in essence, a 
deformable mirror surface whose shape can be digitally 
controlled. The mirror is also fast. Each element can be 
deflected at an 8 kHz. rate, although the current 
experimental setup operates the DMD at 50 Hz. to maintain 
compatibility with a 50 Hz. European standard video. 

At the present, the DMD is being studied for use as an 
optical correlator. Correlation strength measures can be a 
key feature in vision identification tasks [2]. 

Correlation, however, has not been normally used in 
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practical vision systems since it is computationally 
intensive and slow. This is not the case using the DMD where 
correlation using optical processing is attractive. The two 
classical approaches to correlation using optical processing 
are Vender Lugt Filtering (VLF) and Joint Correlation (JC) . 
The use of VLF has been one of the prime thrusts of previous 
research on the DMD and is described elsewhere [3] . The JC 
approach was developed and studied this summer as a part of 
research conducted under the ASEE summer faculty program. 

The key results of this work are described here. 

The basic approach to JC involves two Fourier transform 
operations and one square law (intensity detection) 
operation. First two objects to be correlated are optically 
Fourier transformed in the same operation. This is done by 
placing the objects side by side in front of a lens and 
transil luminating them with coherent light. The lens 
produces a joint Fourier transform of the objects in terms 
of the electric field 15] . The field is then detected with a 
square law detector. The squaring operation produces 
crossterms containing the Fourier transform of the joint 
correlation of the objects. An additional Fourier transform 
of the squared field (using another lens) will yield two 
crosscorrelation terms. Optically this results in 
autocorrelation peaks in the form of intense points of light 
when the objects are matched. The degree of correlation as 
well as the details of the object structure determine the 
"sharpness" of the correlation point. The implementation of 
the JC process on the DMD required some modifications of the 
traditional JC process. In conventional optical 
implementations the square law operation is usually carried 
out by detecting the intensity of the joint Fourier 
transform on photographic film. A positive transparency of 
the transform then serves as an input for the second Fourier 
transform operation using a lens that yields the JC. In the 
approach used here the film is replaced by the DMD. The DMD 
is not primarily an amplitude modulating device; it is for 
the most part, a phase modulating device. In this case, the 
intensity was phase encoded. 

In previous research, using a computer simulation [5], 
it was shown that directly writing the intensity as a phase 
would yield good quality correlations. This was attempted 
initially by writing the squared Fourier transform patterns, 
calculated by a computer, directly on the DMD. The DMD was 
then illuminated with collimated light from a helium neon 
laser. The reflected light was then Fourier transformed with 
a 50 cm. lens. The resulting inverse transform was then 
imaged on a CCD array camera and observed on a monitor. 

These first attempts were negative and no correlation peaks 
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were observed. The process was reviewed for flaws. One of 
the major problems in the previous computer simulation is 
the assumption that the DMD could be accurately modeled 
using simple reflecting surfaces whose mechanical deflection 
was directly proportional to the charge pattern written into 
the DMD. In the real device, this does not appear to be the 
case. Based on data obtained by TI [61, it appears that 
there is a nonlinear relation between charge patterns 
written on the DMD and the resulting pattern of mechanical 
deflection. Therefore, an alternative approach was tried 
that appeared to be much more robust. 

The key identification feature in the NASA tasks is a 
sharp autocorrelation peak that shows strong correlation 
between an object to be identified and a reference object. 

A sharp peak is due primarily to a strong single frequency 
sinusoidal component occurring in the joint Fourier 
transform. The strength of this single frequency component 
is proportional to the number of equally spaced point pairs 
on the objects being correlated. If this sinusoidal 
component can be emphasized in the case of a good match, 
then the correlation spot will be bright. A simple approach 
that appears to accomplish this, and completely avoids the 
nonlinearity of the DMD, is to threshold the modulus squared 
of the Fourier transform and rewrite the squared transform 
as a binary object. If the sinusoidal component is present, 
it will be emphasized as the fundamental component of a 
square grating. Typical results using this approach are 
described next. 


FINDINGS 

The use of JC on the DMD was tried using thresholding. 
The Fourier transform of test object pairs were calculated, 
squared and then thresholded by replacing intensities above 
a constant value (the threshold) by a 1 and intensities 
below the threshold by 0. In the cases studied, the < 
threshold was set at one percent of the peak value. The 
resulting pattern was written on the DMD. Then an inverse 
Fourier transform was carried out optically and the 
resulting "correlations" were recorded. Quotes are used 
here to indicate that correlation is used loosely since a 
nonlinear mapping has replaced the joint Fourier transform 
intensity with a binary object. 

Figs. 1 and 2 show two sets of objects that were used 
to test the DMD as a recognition correlator. Fig. 1 shows 
the pair SS and Fig. 2 shows the pair ST. In this case, the 
S and T can be considered as reference objects that are 
being compared with a test object, i.e., another S. A sharp 
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Fig. 1. Tha tast pair SS , 
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Fig. 2. The t*st pair ST. 
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correlation peak is desired when the objects are matched to 
indicate recognition. However when the objects are not 
matched then a weaker peak (or a peak too weak to observe) 
is desired. 

Figs. 3 and 4 show the computer generated Fourier 
transforms after threshholding and converting to binary 
images for the SS and ST pairs, respectively. Note the 
strong single spatial frequency component for the SS 
transform in the diagonal direction. This frequency 
corresponds to the separation between the objects and will 
determine where the correlation peak will appear in the 
output plane. Figs. 5 and 6 show respectively, the 
resulting correlations made using the DMD. The images were 
taken from a CCD array camera and stored using a frame 
grabber. Note the presence of strong off-axis correlation 
peaks in the case of a match as shown in Fig. 5 and the lack 
of distinct peaks in Fig. 6. It should be pointed out that 
all the figures shown are only crudely grey scaled using a 
10 level grey scale plotter in conjunction with a laser 
printer. The grey scale code is shown below the figures. 

CONCLUSIONS 

The results presented suggest that the DMD can be used 
to correlate images for identification. with good results 
provided that proper thresholding is used. Since only a 
small set of objects was tried it is not possible to offer a 
general scheme for thresholding. It is not known how well 
the particular threshold scheme used here will work in 
general. Furthermore, more testing is needed to evaluate 
scaling, rotation and noise effects using a more interesting 
object set. 

Also, the similarity between the thresholding approach 
used here and in optical neural networks has not gone 
unnoticed. Indeed, the only element missing to convert this 
system to an optical neural net system is a feedback 
mechanism. This is being considered in future studies using 
a system the author is presently developing (7] . 

ADDITIONAL COMMENTS 

During the preparation of this final report, but after 
the author's ASEE summer epoch, some further results were 
obtained on the JC during a joint "overnight research 
adventure at NASA JSC " involving Richard Juday and a summer 
intern student from The University of New Hampshire ,Joe 
Bailey. The DMD was tested using photographic 
transparencies as the correlator input. The squared Fourier 
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Fig. 3. The computer generated Fourier 

transform of SS after thresholding. 
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Fig. 4. The computer generated Fourier 

transform of ST after thresholding 


s?S£ SSW 


15-10 


ORIGINAL PAGE IS 
OF POOR QUALITY 



Fig. 5. The joint correlation of the 
SS pair. 
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Fig. 


. Th« joint eorralation of tha 
pair ST . 
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transform was obtained optically using a lens and directly 
recorded on a CCD camera. The camera image was written into 
the DMD using a frame grabber. Good quality correlations 
were obtained. These results will be reported in detail at a 
later date. It would appear that real-time JC is possible 
provided a fast spatial light modulator (SLM) is used as an 
input to the DMD correlator. Such a real-time device is in 
the process of being installed. The only additional 
requirement is a fast frame grabbing routine for 
transferring the Fourier transform intensity to the DMD. 

This should not be a problem at the 50 Hz. frame rate now 
being used to write to the DMD. 
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